(1) Field of the Invention
The present invention relates to a system for non-linear video editing in which a video is divided into shots and after video effects have been applied, the shots are rearranged to produce a new video.
(2) Description of the Related Art
FIG. 1 is a block diagram showing the structure of a conventional video editing system. The system includes a VTR 101 having a source tape, a VTR 102 having a master tape, a sync signal generation unit 103 for generating signals to synchronize the VTRs 101 and 102, a shot link information memory 104 for holding information on the linking order of shots set by the user, and an edition control unit 105 for controlling the operations of the VTRs 101 and 102 in accordance with the information.
FIG. 2A shows a source tape 201 having Shots A, B and C corresponding to frames 101-193, 250-289, and 320-416 respectively, and a master tape 202 on which Shots C, A, and B are rearranged to correspond to frames 1-97, 98-190, and 191-230 respectively. FIG. 2B is a shot information table showing selected shots Nos. and start frame Nos. and termination frame Nos. in the source tape 201 and the master tape 202.
FIGS. 3A and 3B shows a flow chart depicting a series of operations of the edition control unit 105 for dubbing Shots A, B, and C on the source tape 201 onto the master tape 202 as shown in FIG. 2A, based on the shot information shown in FIG. B.
(a1) The operation is started when the user inputs a direction to the edition control unit 105.
(a2) All the shot link information shown in FIG. 2B is inputted to the edition control unit 105.
(a3) The shot No. shown in FIG. 2B is initialized to 1.
(a4) The start frame 320 for shot No. 1 on the source tape 201 is accessed.
(a5) After the completion of the access, the source tape 201 is rewound a little bit further than the accessed frame. This is called pre-roll, which allows the source tape 201 to reach a normal replay speed by the time the accessed frame starts to replay.
(a6) The start frame 1 for shot No. 1 on the master tap 202 is accessed.
(a7) After the completion of the access, the pre-roll is executed.
(a8) The VTR 101 having the source tape 201 starts replaying.
(a9) The VTR 102 having the master tape 202 starts recording.
(a10) The timing of the replaying of VTRs 101 and the recording of 102 is adjusted for synchronization with reference to the synch signal sent from the sync signal generation unit 103.
(a11) The adjustment is continued until the synchronization is achieved. This process is called phase modification.
(a12) After the completion of the phase modification, it is checked whether the master tape 202 is ready to start from the start frame 1.
(a13) After the check, the VTR 102 having the master tape 202 is set on a recording mode.
(a14) It is checked whether the VTR 102 has recorded up to the termination frame 97.
(a15) When the recording is completed, the recording mode is cancelled.
(a16) The VTR 101 for the source tape 201 is stopped.
(a17) The VTR 102 for the master tape 202 is stopped.
(a18) It is judged whether the shot No. in process is the largest.
(a19) If it is not the largest shot No., the shot No. is incremented by "1" and goes back to (a4) for processing the next shot.
These steps (a1)-(a19) are repeated until all the selected shots A, B, and C are dubbed onto the master tape 202 as shown in FIG. 2A. The steps (a8) and (a16) may be executed simultaneously with the steps (a9) and (a17) respectively.
The following is an operational description when two source tapes are used to create a master tape.
FIG. 4 is a block diagram showing the structure of another conventional video editing system, which is provided with two VTRs for source tapes. The system is different from the system shown in FIG. 1 as follows.
There are two VTRs: VTR AC 1011 and VTR BC 1012 for source tapes so that a shot link information memory 1041 takes care of data in both the VTRs. There are a video effects unit 1013 provided for applying video effects, a shot effects information memory 1042 provided for storing information on the video effects.
The edition control unit 105 not only controls operations of the VTR AC 1011 and VTR BC 1012 for source tapes and the VTR 102 for a master tape, based on the cut link information, but reads the contents of the video effects information memory 1042, thereby forwarding them to the video effects unit 1013. The VTR AC 1011 and VTR BC 1012 may be used independently in accordance with the contents of the edition.
Here, "channels" are not limited to frequencies in TV broadcast but represent programs or records. Although AC channel and BC channel are usually produced for different purpose from each other, they may be former half and latter half of a program.
Generally, the video effects are applied to image signals divided into luminance (Y) components and chrominance (C) components.
FIG. 5A shows a source tape 501 for AC channel having Shots A, B, and C and a source tape 502 for BC channel having Shots D and E and a master tape 503 onto which these shots are recorded with video effects called "picture in picture", which will be described below. These source tapes 501 and 502 are hereinafter referred to as AC roll 501 and the BC roll 502 respectively.
FIG. 5B are tables showing shot link information stored in the shot link information memory 1041 for the AC roll 501 and BC roll 502.
FIG. 5C is a table showing information on video effects. In the table, in-points and out-points indicate start frame Nos. and termination frame Nos. respectively on the master tape 503 to apply video effects. The video effects Nos. indicate the types of video effects: PIPACBC indicates "picture in picture" in which a child screen BC is displayed on a parent screen AC as shown in FIG. 6A, and the numeral 10 which follows PIPACBC indicates the contents of the effects. Besides the "picture in picture", the video effects include "wipe" in which a shot is partly broken away and the next shot is projected in the broken-away part so that the former is gradually replaced by the latter as shown in FIG. 6B, "fade in, fade out" in which a shot gradually appear or disappear from a black or white screen, "dissolve" in which a shot gradually fades out to disappear and another shot gradually fades in to appear until the former is replaced by the latter, "DSK" (down stream key) in which letters are superimposed on a picture image, "replacement" in which a shot appearing in another shot grows out of the latter shot until the latter shot is replaced by the former shot, "multichannel synchronous display with multiwindows", and "mosaic" in which a shot is divided into small lattices and the lattices are rearranged.
On the AC roll 501 shown in FIG. 5A, Shots A, B and C correspond to frames 101-193, 250-289, and 320-416 respectively, and on the BC roll 502, Shots D and E correspond to frames 501-620 and 751-860 respectively. The Shots C, A, and B are inputted and rearranged to correspond to frames 1-97, 98-190, and 191-230 respectively on the master tape 503. Simultaneously with the input of Shot C, A, and B, Shots D and E are inputted with the effects "picture in picture" and rearranged to correspond to frames 1-120 and 121-230 respectively on the master tape 503. As a result, on the master tape 503, Shot C is played on the parent screen followed by Shots A and B while on the child screen, Shot D is played followed by Shot E. The speed of driving these rolls are adjusted in the same manner as the case where a single roll is used.
However, it takes time to search the start frame of each selected shot on both the source tapes and the master tape. In addition, both VTRs need time for pre-roll, phase modification, and recording for each shot.
To avoid this, a non-linear image edition apparatus has been proposed, according to which images to be edited are digitalized and then stored to a randomly accessible disk.
Analog/digital conversion of image data in each frame is carried out by dividing the image data into small lattices with a number of vertical and horizontal lines and then digitizing the luminosity of each color of red, blue, and green. In case of audio data, audio signals in each time point are divided into a small range of frequency band to digitize its strength. Digital/analog conversion of image data is carried out in the reversed operations. Furthermore, both vidual and audio digitization has been developed.
Data to be edited, whether they are digital or analog, are usually specified by the storing positions of the starting and ending points, stored in correspondence with the storing position, and read out, based on its positional information.
A digital data storage device capable of randomly accessing based on address information is well known such as Maxtor Corporation MXT-1240S with sector access.
FIG. 7 is a block diagram showing the structure of a conventional non-linear video editing system, which edits digital video data stored in a randomly accessible digital data storage unit.
The system includes an AC channel shot link information memory 71, a BC channel shot link information memory 72, a storage unit 73 for source HD, an AC replay switch 74, a BC replay switch 75, a video effects information memory 76, a video effects unit 77, and a storage unit 78 for a master HD. Other components irrelevant to this invention are not illustrated.
The following is a description of editing Shots A1, A2, B1, and B2 according to this system by applying "wipe" effects shown in FIG. 6B. Shot A1 on the AC channel, which displayed first is gradually replaced by Shot B1 on the BC channel, Shot B1 is then replaced by Shot A2 on the AC channel, and Shot A2 is replaced by Shot B2 on the BC channel.
As shown in FIG. 8, the storage unit 73 for source HD holds Shot A1, A2, B1, and B2 on a single HD. The areas with slanting lines indicate frames excluded from edition. The reference numbers indicate frames adjacent to each Shot. In fact, the digitized image data are concentrically stored on a HD, where each frame corresponds to the circumference; however, it is illustrated in the form of a tape to make it easier to compare with conventional data.
The AC channel shot link memory 71 and the BC channel shot link memory 72 hold information on shots on AC channel and shots on BC channel as shown in FIGS. 9 and 10 respectively. The tables in FIGS. 9 and 10 show selected shots and frame Nos. which designate the start and termination of each shot on each of the source HD and the master HD.
FIG. 11 is a table showing video effects information stored in the video effects information memory 76. In the table, in-points and out-points indicate start frame Nos. and termination frame Nos. respectively on the master HD to apply video effects. The video effects Nos. indicate the types of video effects. The WIPE indicates the video effects "wipe", and each numeral 05, 14, and 07 indicates the types of the wipe.
The storage unit 73 for source HD receives information on the replay of Shot A1 from the AC channel shot link information memory 71 and reads out Shot A1 when the time to replay Shot A1 has come.
The AC replay switch 74 decodes the AC channel replay information sent from the AC channel shot link information memory 71 and is turned on when the time to replay Shot A1 has come, thereby transferring Shot A1 from the storage unit 73 for source HD to the video effects unit 77, and is turned off when the replay is completed.
Then, the storage unit 73 for source HD receives information on the replay of Shot B1 from the BC channel shot link information memory 72 and reads out Shot B1 when the time to replay Shot B1 has come.
The BC replay switch 75 decodes the BC channel replay information sent from the BC channel shot link information memory 72 and is turned on when the time to replay Shot B1 has come, thereby transferring Shot B1 from the storage unit 73 for source HD to the video effects unit 77, and is turned off when the replay is completed.
The video effects unit 77 applies Shots A1 and B1 with the wipe effects for replacing A1 by B1 as shown in FIG. 6B, .based on the video effects information sent from the video effects information memory 76, thereby forwarding to the storage unit 78 for master HD.
In the same manner, the storage unit 73 for source HD receives information on the replay of Shot A2 from the AC channel shot link information memory 71 and reads out Shot A1 when the time to replay Shot A1 has come.
The AC replay switch 74 decodes the AC channel replay information sent from the AC channel shot link information memory 71 and is turned on when the time to replay Shot A2 has come, thereby transferring Shot A2 from the storage unit 73 for source HD to the video effects unit 77, and is turned off when the replay is completed.
The video effects unit 77 applies Shots B1 and A2 with the wipe effects for replacing B1 by A2 as shown in FIG. 6B, based on the video effects information sent from the video effects information memory 76, thereby forwarding to the storage unit 78 for master HD.
Then, the storage unit 73 for source HD receives information on the replay of Shot B2 from the BC channel shot link information memory 72 and reads out Shot B2 when the time to replay Shot B2 has come.
The BC replay switch 75 decodes the BC channel replay information sent from the BC channel shot link information memory 72 and is turned on when the time to replay Shot B2 has come, thereby transferring Shot B2 from the storage unit 73 for source HD to the video effects unit 77, and is turned off when the replay is completed.
The video effects unit 77 applies Shots A2 and B2 with the wipe effects for replacing A2 by B2 as shown in FIG. 6B, based on the video effects information sent from the video effects information memory 76, thereby forwarding to the storage unit 78 for master HD.
As a result of these operations, an edition shown in FIG. 6B is completed and the video thus produced is recorded to the storage unit 78 for master HD, thereby being displayed on CRT or the like.
However, when a plurality of shots are selected and rearranged with video effects according to this non-linear video editing system, the switches must be frequently changed over to keep correct start frames and termination frames. This demands timers and circuits, thereby enlarging the scale of the system resulting in undesirable increase in hardware.
When a device of another type is used as the storage unit 73 for source HD, consecutive replay may be impossible because their storage HDs have different performances from each other in memory density or rotation rate.
In order to read out data from a HD, it is necessary that a read out unit in the storage unit access the HD and transfer the accessed data. Therefore, the speed for transferring data is limited by a time required to access a start frame, a time to wait for a rotation of the disk, and a time to transfer data. As an approach to minimize the limitation for the purpose of synchronizing the data transfer speed with the process speed of other units such as CRT, image data constituting a frame are divided into odd-numbered scan line data and even-numbered scan line data, thereby being stored to different storage disks, which can operate in parallel. The approach allows the transfer speed to increase; however, the performance cannot be optimized with the consecutive replay parameter fixed.
When a plurality of storage disks, one of which is operable, is used, consecutive replay may be hindered by a disk with lower speed. This problem can be solved by providing a temporary storage unit to the lower-speed storage disk; however, this leads to the increase in the cost.
A hard disk or a photo-magnetic disk used as the storage disk often includes defective areas incapable of recording data. The defective areas are replaced by substitutive areas; however, the defective areas are not physically contiguous with adjacent areas, so that unexpected seek time or rotation wait time may be necessary, thereby hindering consecutive replay.
It is desired that the output of edited image is not interrupted by the user's operation to change the contents of the edition.
It is desired that data transfer from different HDs are synchronized and the difference in transfer speed between the HDs are adjusted by applying FIFO, when digital data are transferred from the storage unit for source HD to the video effects unit.
Furthermore, the efficient use of transmission buses for transmitting data or instructions for the data transmission is desired.
Also, speed up and automation of the edition and simplification of the operation are desired.
In addition, cost reduction of the video editing system by reducing both the capacity of FIFO and the expansion of CPU is desired.
To use old analog image for video edition by converting into digital data is desired.
To apply video effects to long hour videos such as movies is desired.
To edit audio information such as radio programs is desired.